OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue

Abstract This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat models, models fulfill at least two task-specific modules: Dialogue state tracker (DST) and response generator (RG). The consists of the domain-slot-value triples, which are regarded as user’s constraints to search domain-related databases. large-scale data with annotated structured usually inaccessible. It prevents development dialogue. We propose a simple yet effective pretraining method alleviate this problem, phases. first phase is pretrain on contextual text data, where information extracted by extracting tool. To bridge gap between downstream tasks, we design tasks: ontology-like triple recovery next-text generation, simulates DST RG, respectively. second fine-tune TOD data. experimental results show that our proposed achieves exciting boost obtains competitive performance even without any CamRest676 MultiWOZ benchmarks.